Summarizing Human Activity in Video∗

We propose a solution for detecting and summarizing human activities from a large amount of video. A video is divided into short segments(called stories), and a set of simple motion/image features are computed for each image frame. Prototypical elementary action patterns(called action events) is constructed by vector quantization on these features.A bipartite graph is constructed by taking stories and action events as nodes, and the co-occurrence relationship between them as the graph edges.We compute an optimal reordering of the video story and the action event nodes so that they are in maximum correlation with each other. The optimal mapping between the stories and action events leads to a grouping and ordering of the video stories into activity patterns. The corresponding ordering on the action events reveals their characteristic features. Experimental results are shown a hospital surveillance video.